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ABSTRACT 

The extent to which colleges are complying with the 
1982-1983 Pell Grant validation requirements was assessed. Fall. 1982 
financial aid data were drawn from a representative sample of 3,490 
Peli Grant recipients at 317 colleges that are part of the Regular 
Disbursement System. Key findings show: (1) the vast majority of 
institutions collect the required verifying documentations for their 
students who are "flagged** for validation by the U.S. Department of 
Education; (2) about 78 percent of the flagged recipients satisfied 
the validation requirements by providing a signed copy of their 
federal taA. return or other verification; (3) the great majority of 
institutions appeared to be identifying incorrect application entries 
in the cases of recipients flagged for validation; (4) for most of 
the documented cases, the application item discrepancies were small; 
(5) about 2 percent of the documented flagged cases had 
out-of-tolerance differences that would lead to a change in the 
student's expected award; and (6) about 6 percent of the documented 
flagged cases had differences within tolerance that would lead to a 
payment change. Appended are study findings and information on the 
sampling methodology. (SW) 
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EXECUTIVE SUMMARY 

/■ 

The purpose of this study is to provide the Department of Education (ED) an 
early measure of the extent to whiCh postsecondary institutions are complying with 
the 1982-83 Pell Grant vaUdation requirements. Data for the study were drawn during 
fall, 1982 from the financial aid files of a statistically representative sample of 3,490 
Pell Grant recipients at 317 institutions on the Regular Disbursement System (RDS).^ 
Key findings show: 

• The vast majority of institutions ooUect the required verifyii^ documen- 
tation for their students who are 'flagged" for validation fay EO. As Figure 
1 indicates, only 6 percent of all flagged recipients did not have the 
required documents in their files as of fall, 1982* 

• Approximately 76 percent of the flagged recipients, as Figure 1 also 
indicates, satisfied the validation requirement by providing a signed copy 
of their Federal tax return; 2 percent submitted an acceptable alternate to 
a tax return; 15 perccfit signed a statement asserting that no tax return 
was, or would be, filed; and 1 percent had no verifying documentation 
because they were exempt from validation, 

• The great majority of institutions appear to be identifying incorrect 
appUcati<m entries in the cases flagged for validation. Adjusted gross 
income (AGI) was correct in 89 percent of the cases and U.S. taxes paid 
was correct in 85 percent of the cases when values from verifying/ 
documents were compared with values on the application, as indicated on 
the flagged recipients' most current Student Aid Report (SAR). / 

• For most of the documented cases, the application item discrepancies were 
small. Only 7 percent of the documented flagged cases had itcrfl 
discrepancies that when taken individually and/or summed exceeded the 
ED-established tolerances. AGI exceeded tolerance in 3 percent of t^ 
cases while U.S. taxes paid exceeded tolerance 3 percent of the time. / 

• Approximately 2 percent of the documented flagged cases had out^f- 
tolerance differences which would lead to a change in the studf(nt*s 
expected award. For the 1.7 million recipients represented by the sarfiple, 
these changes translate into an estimated net overpayment of $3.4 miilion. 
Ibis dollar figure can be viewed as an estimate pf the level of institujiional 
noncompliance as of fall, 1982. / 



Institutions on the RDS disburse Pell funds to their students and are respor 
validation of application information. A small number of institutions, not ref 
in this study's sample, are on the Alternate Disbursement System (ADS). Thes 
do not handle Pell funds and are not responsible for validation. / 



jible for 
resented 
schools 




• Approximately 6 percent of the documented flagged cases had diff erances 
within tolerance which would lead to a payment change. These changes 
translate into an estimated net overpayment of $1.6 milUon as of fall, 
1982. This dollar figure can be viewed as an estimate of the dollar savings 
to the program not captured as a result of the existing validation 
tolerances. 

• PubUc institutions appear to be most (fiiigent in complying with the 
valuation regulations, followed in order by private and p roprietary institu- 
tions. Roughly 3 poicent of the files of flagged students at public 
institutions were ir complete, while 8 percent of the private institution 
files and 19 percent of the proprietary institution files were incomplete. 

• Many institutions are voluntarily taking steps to improve the quality of 
application data. Figure 2 indicates that institutions collected a Federal 
tax return from 28 percent of their unf lagged recipients, an alternate to a 
tax return from 1 percent, and a statement saying no tax return was, or 
would be, filed from 10 percent. A file comparison between the validating 
document and the student's SAR showed that tor these unflagged cases, 
AGI was accurate 89 percent of the time and U.S. taxes paid 91 percent of 
the time. 

• Institutions rarely verified those application itenu that are optional for 
validation. For example, the asset items were documented in less than 0,2 
percent of the flagged cases. 

• Early indicatians are that institutions in 1982-83 are less likely to be out of 
compiianoe with the validation requirements than imtitutions in 1980-81. 
Approximately 11 percent of the flagged recipient files reviewed during 
the 1980-81 Quality Control study had no verifying documentation of AGI. 
In roughly 10 percent of the fUes, U.S. taxes paid was not documented. In 
1982-83, ^ percent had no documentation of AG! and 6 percent were 
missing documentation of U.S. taxes paid. 

Results of this study are limited due to the timing of the data collection. 
Institutions under current validation rules have considerable freedom in making initial 
Pell Grant disbursements to students before completing the validation process. An 
institution that appeare</ to be out-of-compliance when their files were reviewed in 
the fall may, in fact, have chosen to complete validation after their fall 
disbursements; therefore, it is not possible to assert with confidence that an 
institution has made an error until its files are reviewed in the winter or spring of the 
academic year. 

One purpose of the full replication of the 1980-81 Quality Control study, to be 
conducted in spring, 1983, is to answer many of the policy-relevant questions regarding 



the effectiveness of the 1982-83 validation scheme that were left 



unanswered by this narrowly focused study. Among the questions that will be 
addressed in the upcoming study are: 

• How effective was 1982-83 validation in removing payment error? 

• How much payment error and what types of payment error remained after 
validation? 

• What was the burden of validation on the institution? On the stuJent? 
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INCIDENCE OF FILE DOCUMENT ATIONj 
FLAGGED RECIPIENTS 




FIGURE 2 

INCIDENCE OF FILE DOCUMENTATION: 
UNFLAGGED RECIPIENTS 
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INTRODUCTION 

Backgrsuiid 

The Pell Grant Quality Control study of the 19S0-81 award year identified 
Significant overpayments and underpayments that were being made to students as a 
result of misreporting on the Pell appUcation. The findings indicated that student 
misreporting resulted in an average net overpayment of $2*7 per Pell Grant recipient. 
For all recipients represented in the sample, this translated into a net overpayment of 
$2k6 million. Moreover, the study indicated that a substantial portion of this net 
overpayment was due to the misreporting of a single application item— adjusted gross 
income (AGl). 

In response to these findings, the Department of Education (ED) took corrective 
action by greatly expanding the validation requirement under which participating 
postsecondary institutions must verify student application information against certain 
documentatiwi, such as Feder^ income tax returns, before making a disbursement. In 
past years, ED "flagged" for validation the appUcations of only 10 percent of aJl 
eligible applicants. For each flagged application, institutions were required to verify 
the accuracy of at least seven application items. These items included AGI, U.S. taxes 
paid, nontaxable income, household size, number in postsecondary education, 
dependency status, and the dependent applicant's income. For 19g2-g3, ED is selecting 
roughly 60 percent of all eligible applicants for a somewhat less intensive validation 
procedure.^ During the current year, ED requires that schools verify at least two 
application items, AGI and U.S. taxes paid. A third item, parental Social Security 

^ Prior to aune 16, 1982, all eligible applicants were selected for validation. Beginning 
on that date, only applications meeting one of a series of selection criteria (known as 
Pre-established Criteria, or PECs) have been selected. As of October 1982, roughly 13 
percent had been selected according to the PECs and 87 percent had been selected 
randomly. 
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benefits, is required lor validation when student application information does not 
match information from the Social Security Administration (SSA).^ Finally, a fourth 
item, all other income and benefits, is required if certain line items on the collected 
Federal ux return conflict with information on the application. 
Purpose and Limitations of Study 

The purpose of this report is to provide ED with a preliminary asiessm«it of the 
19S2-83 validation procedures. Specifically, the preliminary study was designed toi 

• Measure the extent to which instituticMW are complying with the new 
validation requirement; and 

• Identify areas Of ncMKompliance. 

This study does not present a thorough evaluation of the 1982-83 validation 
scheme. The following are amoTig its limitations} 

• Data were collected for the assessment during fall, 1982, an early point in 
the Pell Grant processing cycle. During 1982-83, institutions have 
considerable leeway in making disbursements to flagged students. 
Institutions, at their own ri^, may make first disbursements to flagged 
applicants before collecting verifying documents. In fact, roughly 22 
percent of the institutions polled in e study sample said that they 
followed this {M-actice either on a regular basis or occasionally. Therefore, 
until the second disbursement has been made (normally in the winter or 
spring) it is not possible to assert with confidence that an institution has 
made an error. 

• The study was unable to measwe several possible effects of the 1982-83 
validation scheme. For example, a student with errors who had been 
notified that his cw was selected for institutional validation could have 
responded in one of three ways: 

— The student, knowing he seriously understated income or overstated 
expenses and deductions, drops out of the system. 

The student corrects his flagged Student Aid Report (SAR) 
immediately and arrives at his chosen institution in the fall with a 
SAR containing correct data for income, expenses, and family 
composition. 

— The student is told by the school, after he has brought in 
documentation, to submit corrections to the processor. The 
corrected SAR is in the file by the time the data are collected for 
the preliminary study. 



The Pell Grant application processor matches the Pell application computer tape with 
the 5SA tape. It is expected that roughly 26,000 eligible applicants will be required to 
verify parental Social Security benefits during 1982-83. 




In thwM three cases, validation has had the desired effect of preventing 
erroneous payments, but this study was not able to measure the error 
removed under each of these three situations. 

e This preliminary study, unUke the i9Sa-Sl Quality Control Study, does not 
measure the level of student payment error. Only a few data items were 
verified during the data collection with documents whose authenticity was 
not ensured. 

e Neither does tiiis study adec^tely measure the burden of the validation 
requirement on students and institutions. Appendix C provides anecdotal 
evidence collected from financial aid administrators regarding the burden 
of yaUdation. In addition, from data collected to create a student sample 
for the spring, 1983, replication of the 1980-81 QuaUty Control study, it 
can be estimated that roughly 15 percent of the eligible Pell applicants 
expected to be paid by spring, 1983, had not In fact, been paid at the time 
of the fall site visits. This 15 percent, however, does not necessarily 
reflect a backlog of unpaid students created solely by validation. 
Anecdotal evidence suggests that late delivery of the final 1982-83 Ml 
Grant Payment Schedide and Validation Handbook may have also 
contributed to this hacklog of unpaid students. Moreover, many of these 
students in the •Expected to be paid" category were likely recent 
applicants or , registrants at their institution who had not as yet been 
affected by validation. 

In the longer rim, the Department needs more detail<^d information concerning 

the cost-effectiveness of the new validation scheme- The full replication of the 1980- 

81 QuaUty Control study, the findings of which will not be available until fall, 1983,. 

will provide much of the information needed to fully evaluate the 1982-83 validation 

initiative. The last section of this report shows how the results of this limited stiKiy 

and the results of the full replication fit into a .broader research agenda for validation 

issues. 

Description of the Data 

The findings presented in this report are drawn from sample data collected for 
the 1982-83 Pell Grant program year. The sample was selected to be representative of 
the Pell Grant recipient population attending schools on the Regular Disbursement 
System (RDS)^ as of fall, 1982.* The data for the analysis come from 3,<»90 student 



Students attending institutions on the RDS receive funds directly Irom the 
institution. Institutions on the RDS have responsibility for validation. A small number 
of institutions are on the Alternate Disbursement System (ADS). ADS institutions 
were excluded from the preliminary study sample since they are not responsible for 
validation. 

Details on the sample design are found in Appendix B. 




record abstracts drawn from the financial aid files of the 317 institutions the 
recipients attend. The data were collected in Novem'oer and Decen^ber, 1 9S2. Figure 
1-1 presents a profile of the study sample by dependency status and whether or not the 
sampled cases were flagged for validation. 
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Dependent** 



Total 



Cases Flagged 
For Validation 



798 
23% 



1,317 
38% 



2,115 
61% 



Cases Not Flagged 
For Validation 



18% 



726 
21% 



1,373 
39% 



TOTAL 



1,<>47 
<fl% 



2,0*3 
59% 



3,<f90 
100% 



•Independent recipients receive little w no financial support from their parents; 
therefore, the parents' income and household information is not used in calculating 
the independent recipient's need lor a Pell Grant. 

** Dependent recipients recieve significant finarK:ial support from their parents; 
therefore, the parents' income and household information is used in calculating the 
dependent recipient's need for a Pell Grant. 



FIGURE 1-1 
PROFILE OF STUDY SAMPLE 
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FINDINGS 



This chapter presents the findings of the preUminary study. The following 

specific research questions were f<M^ulated in ollaboration with senior OSFA 

officials in order to meet the study's objectives} 

1. How many recipients flagged for validation had complete file 
documentation? 

2- What type(s) of documentation did instititions collect? 

3. How does tlw iiKudence of file documentation in 1982-83 compare with the 
incidence of docwnoitation in 1980-81? 

For recipients flagged for validation with file ckycumentation, what were 
the differences between the docwncHit valtns and the application values, as 
indicated on the most on-rent SAR, for AGI, U.S. taxes paid, and Social 
Security beswf its? 

9. How many recipients flagged for validation were within validatitm 
tolerances for the tfwee items and for the sum of the three items? 

6. How does the rate at which flagged recipients are out-of-tolerance in 
1982-83 compare with the rate in 1980-81? 

7. For recipients flagged for validation, how often were items other than AGI, 
U.S. taxes paid, and Social Security benefits docimiented? How often did 
documented values not agree with SAR values? 

8. For recipients flagged for validation, what are the Student Aid Index (SAI) 
changes and potential payment consequences (?ward errors) associated with 
the documented discreparK:ies? 

9. What were the characteristics of the institutions having files with 
incomplete documentation? What were the characteristics of the 
institutions that awarded flagged ^students who had out-of-tolerance 
discrepancies? 

10. How many recipients not flagged for validation by ED were validated by 
the institution nevertheless? What were the results? 

In the r'^mainder of this chapter, the findings as they address each of the ten research 

questions are presented and discussed. 



Question 1: How many recipients flagged for vaUdation had complete fUe 
documentation? 

At a minimum, ED requires institutions to coliect a signed copy of the 
independent appUcant's or dependent appUcant's parents' tax return. If the student or 
parents did not or plan not to fUe a tax return, a statement from the student (and 
parent) attesting to that fact is required. When a tax return has been f Ued but cannot 
be obtained, the institution, under certain conditions, may accept alternate documents 
such as a transcript from th<? IRS, a state tax return, or W-2 forms. As a last resort, 
an institution may accept a statement from the student of parent certifying the 
amoL-* of earned income, its source, and why a tax return or alternate document is 
not avaUable. For a small percentage of dependent applicants, ED requires that the 
institution collect a statement from the Social Security Administration (SSA) verifying 
the amount of benefits the student's parents received. 

When each sampled recipient's student aid file was reviewed, a determination 
was made as to whether the documentation was "complete" or "incomplete." A file 
was considered incomplete if either (1) required documents were missing or (2) 
documents were present but they failed to meet the minimum standards set forth in 
the PteU Grant Validation Ham&ook 19«2-«3.^ For example, a file was considered 
incomplete in each of the following situations: if the tax return was unsigned, if the 
statement of nonfiling was not signed by the proper person(s), if the tax return was 
completely illegible, or if a copy of only one side of the two-sided 10^0 tax return was 
present in the file. 

The data in Figure 2-1 indicate a high level of compUance among institutions 



Financial aid administrators are not obliged to go beyond what is required by the 
VaUdation Handxiok to determine the authenticity of the tax return copies and other 
documentation. Likewise, an independent effort to authenticate the fUe 
documentation was beyond the scope of this study. In the Pell Grant Quality Control 
Study of the 1980-81 academic year, a comparison of tax return data from the IRS 
with tax return data in student aid files showed that file data for AGI was incorrect 8 
percent of the time. 



Cases Fercent 

Total Recipients Flagged for Validation 2,115 100% 

Exemptions 29 1% 

Complete Documentation 1,956 93% 

Incomplete Documentation 130 6% 



FIGURE 2-1 

INCIDENCE OF FILE DOCUMENTATION FOR PELL GRANT 
RECIPIENTS FLAGGED FOR VALIDATION 
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with this validation requtr»nent. According to the figure, only 6 percent of flagged 
reopients (fid not hav« the required documentation in their files as of fali, i9S2, a 
relatively early point in the Pell processing cycle. Complete documentation had been 
collected for 93 percent whiie 1 percent met at least one of seven conditions under 
which the student was exempted from validation.'^ 

Note that the 6 percent with incomplete file <kx:umentation does not necessarily 
reflect institutional noncompliance. Under the existing validation provisions, 
institutions are granted considerable leeway in making initial disbursements to flagged 
applicants. An instituticm has the option of making a disbursement of up to one-half 
the amount of the stiKienfs expected award for the entire academic year before 
collecting any verifying documentation. (The institution exercises this option at its 
own risk since it is liable for any overpayment that is discovered following validation.) 
Therefore, until more than one-half of the student's awau'd has been disbursed— 
normally occurring with the second disbursement in the winter or spring of the 
academic year— it is not possible to assert that an institution has not complied. 

Question 2: What type(s) of documentation did institutions collect? 

Figure 2-2 divides flagged cases according to whether Social Security benefit 
documentation was required (column 2) or whether it was not (column 0. The results 
indicate that for the few cases where Social Security benefit documentation was 
required, a tax return was rarely missing but that financial aid officers often 
neglected to collect a statement from the SSA. Anecdotal information collected 
during the field work (see Appendix C) suggest that many aid officers are having 



If an aid administrator can document that the flagged student is a legal resident of 
one of the U.S. Territories, has filed a Special Condition Application, has completed 
validation at another school, is incarcerated, has parents who reside in and are citizens 
of another country, is a recent refugee to the United States, or is dead, that student 
need not go through the validation process. 
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Cases Flagged 
for Tax 
Return 



Cases Flagged 
for Tax 
Return 
and SSA 
Statement 



Total 
Flagged 



Recipients Flagged for Validation 



Percent Exempt 



2,093 



1% 



22 



5% 



2,115 
1% 



Percent with Complete Documentationj 

- With Tax Return 

- With IRS Transcript 

- With State Tax Return 

- With W-2 Forms 

- With Statement from Student/ 
Parent 

- With Statement of Nonfiling 



76% 
1% 



15% 



32% 



76% 
1% 



15% 



Percent with Incomplete Documentation: 

- Missi.ig Tax Return or Acceptable 
Alternative 

- Missing SSA Statement 

- Missing Both 



6% 9% 
Not Applicable 27% 
Not Applicable l<f% 



6% 



•Less than 0.5% 



nCURE 2-2 

INCIDENCE OF FILE DOCUMENTATION FOR PELL GRANT 
i' JPIENTS FLAGGED FOR VALIDATION BY TYPE OF CASE 
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difficulty validating Social Security benefits either because of confusion over program 
regulations or because of lack of cooperation from the SSA. / 

Figure 2-2 also details the types of verifying documents that institutions collect 
to satisfy the minimum validation requirement. As the figure indicates^ 76 percent 
sulMnltted a signed copy of a tax rctiaii wliUe 15 pe^^ent submitted a statement 

/ 

/ 

attesting that a tax re«m was not or would not be ^^iled. For a very anall portion, a 
tax return was filed but could not be obtained so an alternate document was used for 
validation. A transcript from the IRS itemiring AGI and U.S. taxes paid was the most 
commonly submitted alternate document. The least reliable acceptable alternate 
document, a written statement from the student <x parent attesting to the amount of 
income and Fe<teral taxes paid, was collected for less than 0.3 percent of the cases. 

Question 3i How does the incidence of file documentation in 19S2-83 compare with 
the incidenoe of documentation in 1980-81? 

As part of the Pell Grant Quality Control study of the 1980-81 academic year, 
student financial aid files for a representative sample of institutions were reviewed in 
spring, 1981. Figure 2-3 compares the percent of recipients in 1980-81 with the 
percent in 1982-83 who had rK> file documentation for tte two most criticzi validation 
items, AGI and U.S. taxes paid. The data suggest that institutions in 1982-83, when 
ED is requiring vaUdation for rcughly 60 percent of aU recipients, are far more 
diligent in collecting the required documents than they were in 1980^81, when ED 
required validation for roughly 10 percent of all recipients. Note that the comparison 
is not perfect. Files for 1982^83 were reviewed in the fall and files for 1980-81 were 
reviewed in the spring. One would expect that the percentage of files with incomplete 
documentation will <iecrease when the files arc reviewed in spring, 1983, for the full 
replication of the 1980-81 study. 

/ 

Question For recijpients flagged for vaadation with fUe docixnentation, what 
were the differences between the document values and the application 
values, as indicated on the most current SAR, for AGI, U.S. taxes paid* 
and Social Security benefits? 

2-6 
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# 



Percent of Redptents With 
AppUcation Item No File Doggncntation 

Adjusted Gross Income (AGI) 11.2% 4.3%»* 

U.S. Taxes Paid 9.6% 5.9%** 



* Note that information here is not directly comparable with information in Figure 
2-2, Data in Figure 2-2 indicate case doci«nentation while this figure indicates 
item documentation. Fcr example, in Figure 2-2 a case may be considered 
incomplete yet have valid documentation for one item and not the other. 

** A statement of nonfiling was considered to be valid documentation for the purpose 
of this table. 



nCURE 2-3 

COMPARISON OF 1980-81 AND 1982-83 
INCIDENCE OF FILE DOCUMENTATION 
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During each file review, values on verifying documents were compared with 
corresponding values on the sampled recipient's most current SAR. Figure 2-* shows 
the results of this a>mparison for the three critical validation items. ^ The figure 
indicates that ACI was the mo^t often correct of the three, witii only 7 percent of the 
documented cases having diffmmces possibly producing an overpayment and # p erce nt 
having cfifferenoes possibly leading to toKierpayment. The data reveal that for roughly 
78 percent of the documented cases, the sum of the differences between the SAR and 
documented values for the three validation items was less than $3, (This is a net 
figure in which positive and negative dif fereiKes cancel each other out.) 

Question 5: How many recipients flagged for validation were within validation 
tolerances for the three items and for the sum of the three items? 

The strict definition of accuracy in Figure 2-<> (within $5) may distort the 

significance of item discrepancies. In many cases, a difference greater than $5 

between a document value and a SAR value will not alter the size of the student's 

expected award. For this reason, ED has established a tolerance option whereby 

validated students with a minimal likelihood of receiving an overpayment do not have 

to have their inaccurate application information corrected by the Pell Grant 

application processor. If the school finds that no single item difference is greater than 

$300 for dependent students or $200 for independent students and that the sum of all 

differences leading to an overpayment is not greater than $600 for dependent students 

or $^00 for independent students, then the student's SAR does not need to be 

corrected. A case that exceeds tolerances need not be reprocessed if the school opts 

to calculate a new Student Aid Index (SAD for the eligible applicant based on the 

correct information. 



Detailed tables with discrepancy rates expressed in dollar ranges are found in 
Appendix A. 
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Pwamn^ Ptroint of 

Dooansited 

with Farcentof Cases with Totai 

Oilf«r«no«9 Documented D iile fe nc e s Cases 

Ptossibly Caste Pbsaitily with 

LecMfingto Accurate Leadii^to Documen- 

Unden wymcnts Within $3 Overpayments taticn* 



Adjusted Gross Income (AGI) 



89% 



7% 



1,706 



U.S. Taxes Paid 



5% 



85% 



9% 



1,67* 



Social Security E^nefits 



3U% 



38% 



27% 



53 



Net Sum of Differences of 
Three Items 



8% 



78% 



14% 



1,7'»* 



♦Note that information here is not directly comparable with information in Figure 2-2. 
Data in Figure 2-2 indicate case documentation, while this figure indicates item 
documentation. For example, a case might have valid documentation for AGI yet be 
considered incomplete for the purpose of Figure 2-2. 



FIGURE 2-4 



CHXIUMENTED DIFFERENCES BETWEEN SAR VALUES 
AND DOCUMENTED VALUES; CASES FLAGGED FOR VALIDATION 
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Figure 2-5 reveals a tUgti ievei ot compUance among institutions with this 
validation requireniant. RougMy 3 peroant of the lUos with documentation lor ACI 
exceeded the $300 (or $200) toleranoe and had no evidence of an iratitutional SAI 
recalculation} likewise 3 percent of the documented taxes paid cases exceeded 
<ciierance and lacked an 0»uaitionai recalculation. Roughly 4 percent of the cases 
were outside of the $^ (or $^) tolerance when only those differences leading to 
overpayment were summed. Independents, the data indicate, are more likely to be 
out-of-tolerance than dependents. Independents comprise roughly one-third of the 
cases with documentation of at least one of the three items, yet account for nearly 
one-half of the out-of-tolerance cases. The fact that more restrictive tolerances are 
applied to independents may explain this result. 

Question 6s How does the rate at which flagged recipients are out-of-tolerance in 
i982-g3 compare with the rate in 1980-81? 

Figure 2-6 compares the out-of-tolerance rates for AG! and U.S. taxes paid 

discovered in 1980-81 and 1982-83. Note that the comparison is imperfect. The 

tolerance for independents was more restrictive in l98C-8i ($100) than in 1982-83 

($200) and the tolerance for dependents was less restrictive ($500 in 1980-81 compared 

to $300 in 1982-83). Nevertheless, the data suggest that financial aid administrators 

have become far more (filigent since 1980-81 in identifying and acting on ACI and U.S. 

taxes paid discrepancies. 

Question 7t For recipients flagged for validation, how often were items other than 
AGI, VS. taxes paid, and Sodai Security benefits documented? How 
often did documented values not agree with SAR values? 

Institutions are required to use the collected tax return to check the consistency 

of the application item "all other income and benefits'* with respect to certain line 

items on the return. These line items are the in>'^*-*'t and dividends exclusion ami the 

mtaxed portions of unemployment compensation, capital gains, and other pensions and 

annuities. Institutions may also use the tax return to document exemptions, how much 
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Adjusted 
Income (AGl) 

Independent 

Jependent 

Total 



rKXXm Ot OOCiU I IWHWu 

Ofttwwi Exceeding 
Toterances 



Without 
Institution 
Recalculation 
of SAI 



2% 
1% 
1% 



With 
Institution 
Recalculation 
€d SAI 



Percent of 
Documented 



Within 



31% 
65% 
96% 



Total 
WithOocu- 
mentatkm** 



33% 
67% 
1,706 
(100%) 



U.S. Taxes Paid 
Independent 
Dependent 
Total 



1% 
2% 
3% 



♦ 
♦ 
# 



31% 
65% 
96% 



32% 
68% 
1,67<» 
(100%) 



Social Security 

Benefits* »» 16% - 8*% 55 

(100%) 



Sum of Differences of 
Three Items 

Independent 2% ♦ 30% 33% 

Dependent 2% ♦ 65% 67% 

Total m 1% 95% i^7<^^ 

(100%) 

♦Less than 0.5 percent. 
♦♦See footnote to Figure 2-U for guidelines in interpreting this data. 
♦♦♦Only dependent students answer this item on the Pell application. 



FIGURE 2-5 

DOCUMENTED DIFFERENCES BETWEEN SAR VALUES AND DOCUMENTED VALUES 
BY TOLERANCE LEVEL: CASES FLAGGED FOR VALIDATION 
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Percent of OocuniMtcd 
cxoocoinK toicrance 

Validation 

AGI Paid 



i980<8i Independent SlOO 10.2% 10.6% 

Dependent $SO0 8.7% 6.8% 

1982-83* Independent $200 2.0% i.*% 

Dependent $300 !.<»% 2.0% 



* Includes only cases with no institutional recalculation. 



FIGURE 2-6 

COMPARISON OF THE INCIDENCE OF OUT-OF-TOLERANCE 
DIFFERENCES BETWEEN 1980^1 AND 1982-83 
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of the earned incpme was earned by each parent (for a dependent student) or by the 
student and spouse (lor an independent student), dependent student and spouse's 
income, itemized deductions, and medical and dental expenses. Institutions may also 
choose to collect documentation which verifies other appUcatiwi items, such as 
statements from pubUc agencies, financial institution records, statements from a third 
party such as a lawyer or accountant, or statements from the student or parent 
themselves. 

Figure 2-7 shows the incidence of file documentation for selected application 
items and the rate at which document values agreed with SAR values. Several items, 
as the data indicate, were rarely documented. Of particular interest, very few finan- 
cial aid administrators appear to have verified the student's household size and number 
in postsecondary education— two very important items in the eligibility formula which 
were both required for validation prior to the 19S2-83 academic year. However, this 
result may be misleading. Anecdotal information collected during the field work 
suggest that many institutions check the consistency of household size and number in 
postsecondary education against informaticm on institutional aid applications and other 
institutional forms already in the student's file. For the purpose of this study, 
however, institutional formr routinely collected irom all lid appUcants were not 
considered documentation. Instead, the intent was to find out the extent to which 
institutions requested a more reliable form of verification, such as a statement from a 
third party or a certified statement from the student an^i parents. Apparently, few do. 

Figure 2-7 also indicates that aU other income and benefits was the most 
discrepant item with over half the dooonented cases having differences which could 
possibly lead to an overpayment. This suggests that many institutions are not 
verifying the consistency of this item with respect to the four applicable line items on 
the tax return: the interest and dividends exclusion and the untaxed portions of 
unemployment compensation, capital gains, and other pensions and annuities. 

2-13 

27 



A- 


NON-MONETARY ITEMS 


Percsnt Where 
Docunented Value Is: 


Total 




Applicatian ttem 


Less 
than 
SAR 
Value 


Equal 
to 

SAR 
Value 


Greater 
than 
SAR 
Value 


with 
Documen- 
tation 




Household Size 


6% 


9*% 




16 




Number of Ex^ptior^ 


2% 


93% 


5% 


1,59'* 




Number in Postsecondary EdiK::ation 




100% 




8 


R 




Percent Where Docimiented 
Value Less SAR Value Iss 


lOtw 




Application Item 


Zero 
or 

Less 


$1 
to 
S200 


$201 
or 
Greater 


with 
Documen- 
tation 




Father/Student Income Portion 


76% 


16% 


8% 


491 




Mother/Spouse Income Portion 


76% 


13% 


10% 


391 






77% 


9% 


13% 


73 




Dependent Student's Income 


t^7% 


17% 


36% 


123 




Student Social Security 
Educational Benefits 


81% 


13% 


6% 


47 




VA Educationai Benefits 


100% 






7 




All Other Income and Benefits 


29% 




26% 


371 


c. 


DEDUCTIONS 


Percent Where Documented 
Value Less SAR Value Is: 


Total 
Cases 




Application Item 


Zero 
or 
Greater 


-$l 
to 

-$200 


"$201 
or 

Less 


with 
Documen- 
tation 




Medicai/Dental Expenses 


64% 


21% 


1^^% 


263 



FIGURE 2-7 

PRESENCE OF DOCUMENTATION AND DIFFERENCES 
FROM SAR VALUES FOR FLAGGED CASESs 
SELECTED APPLICATION ITEMS* 

♦The following items were documented less than 0.2 percent of the time and therefore were 
not included in this figur*": home assets^ business assets, investment assets, cash and 
savings, dependent student's assets, and unreimbursed tuition. 
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Anecciotal information collected during the field work suggest confusion over this 
validation requirement. Apparently, many financial aid administrators either (1) do 
not understand the requirement or the IRS tax rules, or (2) are not aware of the 
requirement. 

Question 8x For recipients flagged for validation, what were the Student Aid Index 
(SAI) changes and potential payment oonsequenoes (award errors) 
associated with the docum^ted discrepancies? 

For each case with a documented discrepancy, a new SAI was cornputed on the 
basis of the new documented values. Using information on cost of attendance, 
enrollment status, and social security and veteran's educational benefit collected 
from institutions, each stiKient's expected award was recalculated using the 
recomputed SAI. 

Figure 2-8 shows the results of these calculations for independent and dependent 
cases flagged for validation. As the figure indicates, roughly one-fifth of all 
documented cases had discrepancies which when recalculated would lead to SAI 
change. However, for most the change was small and therefore the size of their 
expected payment was lamf fected. Only 8 percent of ail docunvented cases had item 
discrepaiKies whidi would ie^ to a payment chaise. These changes resulted in an 
average net overpayment of $88. For the 1.7 million recipients represented by the 
sample, this translates into an estimated net overpayment of $5 million as of the fall 
of the 1982-83 Pell Grant processing year. 

The reader should be very cautious when interpreting the data in Figure 2-8. The 
$5 million figure does not measure institutional noncompliance because (0 it includes 
in-tolerance discrepancies that institutions are not required to correct; and, (2) it is 
likely that the figure will change before the end-of-year account reconciliation as 
institutions discover the discrepancies and make award adjustments. Neither does the 
$5 million figure measure 1982-83 student error because only a few data items, only 
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Independents Dependents Total 

Total with Documentation 567 1,177 1,74* 

Total Percait with SAI Changes ♦ 10% 25% 20% 

With No Payment Change 6% 16% 13% 

With Payment Changp 4% 10% 8% 

Average Net Payment Ctange*» $ lV^ $ 78 $ 88 

Aggregate Net Payment Change* ♦ $ 1.3M $ 3.7M $ 5.0M 



♦Excludes cases where there was evidence of institutional recalculation of the 
student's SAI in the file. 
♦*A positive change indicates a possible overpayment. 



RGURE 1. -J 



PAYMENT CONSEQUENCES OF APPLICATION DATA DISCREPANCIESj 
CASES FLAGGED FOR VALIDATION WITH SAI CHANGES 



C 



for flagged students, are verified with documents whose authenticity has not been 
ensured. Instead, the $5 mUlion figure should be viewed as a snapshot estimate of net 
overpayments remaining after the verification by institutions of a limited number of 
application items in flagged student files early in the 1982-83 Pell prxKxsaing cycle. 

Figure 2-9 compares the payment consequences of item discrepancies within 
tolerance and the item discrepancies out-of^tolerance for cases where ther«» was no 
evidence that the institution recalculated an SAI. As the bottom panel imiicates, 7 
percent of all documented flagged cases had out-of-tolerance differences. The 
majority of these cases had differences which resulted in no SAI change. <Xrt-of- 
tolerance independents, the data indicate, were more likely than dependents to have 
no SAI change. 

The data also reveal that 2 p erce n t of all docwnented cases out'-of-toleranoe 
differences which resulted in a payment change. For the 1.7 milUon recipients 
represented in the sample, these payment changes translate into an estimated net 
overpayment of $y,¥ million as of fall, 1982. This $3.4 million figure can be viewed as 
the level of institutional noncompliance as of fall, 1982. (Note that this figure will 
change as adjustments are made before the eiKj-of-year account reconciliation.) 

The top panel of Figure 2-9 considers the cases with documented differences 
within the validation tolerance. As the data indicate, 6 percent of all flagged 
recipi«its with docianentation had in-tolerance discrepmcies which resulted in 
payment changes. These changes, when weighted to the 1.7 million recipient pool as 
of fail, 1982, translate into an estimated net overpayment of $1.6 million. This figure 
can be interpreted as the dollar savings to the program not captured as a result of the 
existing validation tolerances. 

Question 9: What were the characteristics of the institutions having files with 
incomplete doonnentation? What were the characteristics of the 
institutions that awarded flagged students who had out-of-tolerance 
discrepancies? 
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Independents 



Dependents 



Total 



Total with Documentation 



567 



1,177 



1,7** 



Percent With In-Toierance 
Diiferencess 

Total With SAI Change* 

- With No Payment Change 

- With Pajrment Change 

Average Net Paym«it 
Change** 

Aggregate Net Payment 
Change** 



S% 
5% 
3% 

$ (7) 

$ (0.5M) 



Percent With Out-of-Tolerance 
Differences: 

Total* 

- With No SAI Change: 
With Zero SAI 

With Positive SAI 

- With SAI Change: 

With No Payment Change 

With Payment Change 

Average Net Payment 
Change** 

Aggregate Net Payment 
. Change** 



9% 



6% 



1% 
1% 

$ <f39 
$ i.3M 



22% 

$ « 
$ i.6M 



5% 



2% 



1% 
2% 

$ 231 
$ 2.1M 



17% 
U% 
6% 

$ 33 

$ 1.6M 

7% 
3% 



1% 
2% 

$ 2Stt 

$ 3.^^^! 



♦Excludes cases where there was an institutional recalculation of the student's SAL 
•*A positive change indicates a possible overpayment; a negative change indicates a 
possible underpayment. 

FIGURE 2-9 

PAYMENT CONSEQUENCES OF APPLICATION DATA DISCREPANCIES: 
CASES FLAGGED FOR VALIDATION BY IN-TOLERANCE AND OUT-OF-TOLERANCE 
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Of the 317 institutions in the sample, 200 are pubUc, U are private and non- 
profit, and 33 are. proprietary institutions. The distribution of types of institutions as 
opposed to control is as follows: 21 are less than two-year institutions. 111 are two- 
year institutions, and 1S5 are four-year or more than four-year institutions. Figure 2- 
10 reports the differences in the incidence and payment consecpences of institutional 
noncompliance at public, private, and proprietary institutions. The fincfings suggest 
that public institutions are most diligent in ocm^ying with validation r«quiranients 
while proprietary institutions are the least* Roughly 5 percent of the sampled flagged 
cases at public institutions had incomplete documentation while 19 peiccnt of the 
flagged files at the relatively few proprietary institutions in the sample were 
incomplete. Only 1 percent of the recipients with documentation at public schools had 
out-of-tolerance differences resulting in a payment change while the figure for 
proprietary institutions was 7 percent. Moreover, recipients at public institutions, who 
make up three-fourths of the population with verifying documentation , accounted for 
one-half of the net payment change and recipients at proprietary institutions, who are 
only 3 percent of the population, accounted for one-fourth of the payment error. 

Figure 2-11 stows the distribution of noncompliance by less than two-year, two- 
year, and four-year institi;cions. The findings si^gest that four-year institutioiK are 
most compliant and twc>-year and less than two-year schools are least compliant. 
Four-year sc|xx>ls, the data indicate, seldom had files with incomplete documentation 
and rarely made payments to students with out-of-tolerancc discrepancies. Recipients 
at these institutions make up three-fourths of the population with verifying 
documentation yet accomted for only one-half of the total net payment change. 

Question 10: How many recipients, not selected for validation by ED, were validated 
by the institution nevertheless? What were the results? 

An institution may choose to validate eligible Pell applicants not selected for 

validation by ED. The data in Figure 2-12 suggest that many institutions are 
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/ 



/ 





PubUc 


Private 


Proprietary 


Total Recipients 








I'erOTnt With Incomplete File 
Documentation 


5% 


S% 


19% 


Total Files With Verifying 
DociMnentation 


1,340 


m 


61 


Percent With Out-of-ToleraiKe 
Differences Resulting in Payment 
Change 


1% 


2% 


7% 


Average Net Payment Change 


$ 22t^ 


$ 265 


$ 522 


Aggregate Net Payment Change 


$ K8M 


$ O.SM 


$ 0.9M 


Percent of Total Net Payment Change 


51% 


23% 


26% 


Percent of Population With 
Verifying Documentation 


77% 


20% 


3% 



FIGURE 2-10 

INCIDENCE OF NONCOMPLIANCE AT PUBLIC, 
PRIVATE, AND PROPRIETARY INSTITtmONS 



Less than 2-Y«ar 2-Y«ar ^Year 



Total Recipients *3 509 1563 

Percenf With Incomplete File 16% 10% 5% 

Documentation 



Total Files With Verifying 34 406 1,304 

Documentation 



Percent With CXit-of-Tolerance 3% 2% 2% 
Differences Resulting in Payment 
Change 

Average Net Payment Change $ 100 $486 $ 200 

Aggregate Net Payment Change $0.04M $1.7M $1.7M 

Percent of Total Net Payment Change 1% 49% 49% 

Percent of Population With 2% 23% 75% 

Verifying Documentation 



FIGURE 2-1 i 

INCIDENCE OF NONCOMPLIANCE AT LESS THAN TWO-YEAR, 
TWO-Y*^ aR, AND FOUR-YBAR INSTITUTIONS 
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"7 

*- / 

/ 
/ 

/ 

/ 
/ 



Cases Percent 

Recipients Not Flagged for Validation 1 , 375 100% 

Complete Docunientatiom 

- With Tax Return 38* 2S% 

- With IRS Transcript 9 1% 

- With State Tax Retvm 1 ♦ 

- With W-2 Forms 2 » 

- With Statement from Student/ Parent 2 ♦ 

- With Statement of Nonfiling 139 10% 

Missing or Incomplete Documentation 838 61% 
♦Less than 0.5 percent. 



nCURE 2-12 

INCIDENCE OF FILE DOCUMENTATION FOR 
PELL GRANT RECIPIENTS 
NOT FLAGGED FOR VALIDATION 
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voluntarily t^dng steps to improve the quality of appUcation datai nearly percent 
of the unf lagged cases in the sample had c omple te validation doc u ment a tion In their 

files.^ The tax return, the data indicate, was the most often collected document while 
the IRS transcript was the most common alternative to a tax return. 

Figure 2-13 shows how often file documentation for AGl, VS. taxes paid, and 
Social Security benefits disagreed with the information on the student^ most recent 
SAr7 The data suggest that, once documoitatlon has been collected, institutions are 
as diligent in identifying item corrections for unflagged cases as they are for flagged 
cases. In S9 percent of the documented unflagged C&ses, the document and 5AR values 
for AGI were accurate within $5. In 91 percent of the cases, U.S. taxes paid was 
accurate within $5. 

Figure 2-1 shows the payment consequences of these item discrepancies. The 
data reveal that 17 percent of all unflagged documented cases had differences which 
resulted in an SAI change. For most of these cases, however, the change was 
inconsequential; only 6 percent of all unflagged documented cases had discrepancies 
which resulted in a payment change. Unflagged dependents, the data indicate, were 
far more likely than independents to have item discrepancies which resulted in a 
payment change. Moreover, item discrepancies f<H- dependents resulted in an average 
net overpayment of $153, while independent discrepancies resulted in a net 
underpayment of $11<».* 



See the discussion under Question 1 for the definition of "complete documentation" 
used in this study. 

^ Detailed tables with discrepancy rates expressed in dollar ranges for the three 
validation items and for other application items are found in Appendix A. 

g 

Interpretation of these findings are subject to the same limitations as the data in 
Figure 2-8. See discussiwi under Questicm S. 
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Total 



Item 



Pwcent oi Rwcent of 

Documented Documented 

witn rd vBiik OX v.asw witn 

DocMmented Differences 

Pmatbtf Cases Possibly with 

Leading to Accurate Leading to Documen- 

Upderpayments Within $3 Overpayments tation 



Adjusted Gross Income (ACI) 



89% 



7% 



U.S. Taxes Paid 



5% 



91% 



Social Security Benefits 



50% 



25% 



25% 



16 



Net Sum of Differences of 
Three Items 



8% 



82% 



11% 



FIGURE 2-13 



DOCUMENTED DIFFERENCES BETWEEN SAR VALUES 
AND DOCUMENTED VALUESs 
CASES NOT FLAGGED FOR VALIDATION 
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Indtp^ndents Dwpandits Total 

Total With Documentation 172 271 M 



Total Percent With 5AI Changes* 10% 22% 



17% 



- With No Payment Change 5% 15% 11% 

- With Payment Change yx* 6% 6% 

Average Net Payment Change* ♦ $(11<^) $ 153 $ 61 

Aggregate Net Payment Change* ♦ $( 0 . ) $ 1 . 1 M $ 0 . 07M 



♦Excludes cases where there was evidence of an institutional recalculation in the 
student's file. 

♦♦A positive change indicates a possible overpayment; a negative change indicates a 
possible underpayment. 



nCURE 2-U 

PAYMENT CONSEQUENCES OF APPLICATION DATA DISCREPANCIES! 
CASES NOT FLAGGED FOR VALIDATION WITH 5AI CHANGES 
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CHAPTER3 

TOWARD A LONG RUN RESEARCH AGENDA FOR VAUDATION ISSUES 



The findings dicusssed in the previpus section can be interpreted as answers to 
specific validation quest! 

A broader set of poUcy Issues Encompassing these questions can be identified as 
falling into three major areass the role of validation, comparisons among alternative 
validation schemes, and the evaluation of a specific vaUdatlon approach. These broad 
issue areas and derivative research questions i;epresent the overall policy research 
context in which the results of this preliminary effort must be interpreted- The 
purpose of the fall study was to gather information about early effects of the 1982-83 
validation scheme. From the outset, the focus of the fall study was necessarily 
narrow. Given the limited <rt)jectives of this effort, it is useful to demomtrate how 
these early results fit into a broader research agenda of vaUd^tion issues. 

What follows is a preliminary research agenda which will be expanded during the 
next few months. 

Preliminary Research /.genda and Data Analysis Plan 

Developing a research agenda and data analysis plan involves four steps; 

1. Enumerate the broad issue areas or major policy questions. 

2. For each major policy question, develop specific rescarchable questions or 
hypotheses. 

3. Develop data collection methods, source*, and procedures required by each 
question or hypothesis. 

Specify measures, statistical procedures, and methods for addressing each 
of the detailed policy questions. 

In the following sections we provide the groundwork for the first three steps. 
The last step, a data analysis plan, wUl follow after the research agenda is complete. 
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I de ntif y ing Broad Policy baues 

There are three broad issues which the Department faces in making decisions 
about validations 

• What is the proper role of validation in Federal Student Aid 
programs? 

• What is the appropriate methodology for comparing and choosing 
among alternative validation schemes? 

• What is the best way to meastre the effects on program error of a 
particular validation scheme once implcmiented? 

The findings presented in this report fall into the third category. The timing of 
the data collection and the resulting nature of the data were such that only a subset of 
the questions imder this category could be answered. 
Developing Researchafale Policy Questions and Hypotheses 

The specific policy questions developed to date include: 
Role of VaUdation 

• How does vaUdati(»i relate to program goals? 

• What are the trade-offs between simplification and validation? 

• What is an acceptable level and distribution of error in the student 
aid programs? 

• What are the alternatives to validation? 
Choosing Among Alternatives 

• What are the features which must be specified in order to completely 
describe a validation scheme? 

How many students should be validated? 
WTu^ students should be selected? 
Which application items should be validated? 
How should these items be validated? 
Who should do the validation? 

s How effective is a particular scheme? 

How much application error is deterred ? 

How much error is removed from submitted applications? 

How much error remains ? 

What are distributional consequences of error deterred, error 
removed, and error remaining? 
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• What arc the costs and burtiens ot alternative validation schemes? 

Student burden 
Institutional burden 
Dt,^tmmtal burden 
Third-party txirden 

Measuring Eliectiveness of a Particular Scheme 

• Are institutions complying with the re<iuirements of the validation 
scheme? 

How many students flagged for validation have complete docu- 
mentation? 

Are schools collecting tax forms or alternative AGI documenta- 
tion? 

• How effective is the scheme in identifying and correcting errors? 

How often does a case involve tax mMvfilersf 

How are (k>cumented errms distributed with respect to 

tolerances? 

How often do out-of-tolerarKie differences lead to payment or 
SAI changes? 

• Are schools requiring students to make corrections? 

What differences exist between documented and application 
values? 

What are associated payment consequences? 

• Do schools cto more than what is minimally required? 

How many non-flagged students are validated? 

How often do schools document items other than AGI, taxes 

paid, and Social Security benefits? 

• How effective is the sdieme in deterring and removing error? How 
much error remains? 

• What are the costs and burdens of the scheme. 

As noted earlier, this preliminary report only addressed a limited number of 
specific policy questions under the third broad issue— measuring effectiveness of a 
particular scheme. For example, the last two questions above remain unanswered. In 
addition, some of the findings can only be considered as early indications. More 
complete measures will be available in fall, 1983, after the spring data collection 
activities are completed. 
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Developing Data Collection Methods, Sources and Procedures 

There are eleven major data collection activities, sources, or procedures. These 
are enumerated and described in Figure >-l. It should be noted that only one of these 
data sources (student's financial aid fUe) was available and used to develop th« findings 
discussed in this report. 
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SouroB/Modc NaturaolData 

Fall 82, File Abstract Review of linancial aid files fort 

1. Presence and types of doctmemation 

2. Documented values of f winula elements 

3. Application values of formula elements 

4. ExemptiiMi from validati(Mi 

5. Extent of differences on formula elements 



Spring 83, Student Record Site visits to various institutional offices: 

Abstracts 

Financial Aid Office 

1 . Presence and types of documentation 

2. IX)cimiented values of formula elem^s 

3. Application values of formula elements 
Expected distHirsement and scheduled awards 

5. Satisfactory academic p>rogress 

6. 6 - month program 

7. ApfM-oved pr(^ram of study 

8. Statement of Educational Purpose 

9. Financial Aid Transcript 

Registrar/Records 

1. Enrollment status/courseload 

2. Satisfactory (M-ogress 

3. Cost of attendance 
k. Half-time enrollment 

Admissions Office 
1 . Citizenship 
2- Transcripts 
3. B»A./B.S» 

Other Offices 

1 . Housing arrangement and cost of attendance 



Spring 83, Institutional Structured interview with financial aid adminis- 

Questionnaire trator: 

1. Institutional procedures 

2. Institutional policies 

a. academic progress 

b. packaging 



RGURE 3-1 

PELL GRANT QUALITY CONTROL DATA SOURCES, 
COLLECTION MODES, AND TYPES OF DATA 
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Source/Mode 



Nature of Data 



c. validation 

d. eligibility determination and benefit 
calctilation 

3. Institutional practices 
Instituticmal perspectives 

5. Institutional costs and burdens 

6. Characteristics of institution and student 
body 

a. calendar 

b. types of programs 

c. credit measurement system 

d. FTE enrollment 

e. student mix 

7. Financial aid office operations and sti.'f f ing 

a. degree of automaticni 

b. docimiented prcK^dures 

c. number, bad<groiA)d, and experience of 
staff 

d. participation in on-going training 

e. staff timover 

f . quality control [M'ocedures 



Winter 83, Parept Qtiestionnaire Face-to-face interview with parent(s): 

1. Values of parental application items (AGI, 
taxes paid. Social Security, other nontaxable 
income, family size, assets, debts, etc.) 

2. Review of documentaticMi for values of 
af^lication items 

3. Problems with application process 
Problems with validation process 



Winter 83, Student Questionnaire Face-to-face interviews with students: 

1 . Values of student application items 

2. Documentation of student application items 

3. Problems with application process 
f^. Problems with validation process 



FIGUR£ 3-1 (continued) 

PELL GRANT QUALITY CONTROL DATA SOURCES, 
COLLECTION MODES AND TYPES OF DATA 
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Source/Mode 

Winter 83, Second Party Sources 



Nature <^ Data 

Record checks witiu 

1 . Internal Revenue Service (AGI, taxes paid) 

2. Financial institutions (saving <:hecking) 

3. Tax assessor records (home value) 



Computed Applicant Records 



Corrections Control Group I: 
Nonsampled Students at 
Selected Schools 



Complete history and transactions for sampled 
students via magnetic tape: 

1 . Initial application values 

2 . Corrections behavior 

3. Final an>llcation values 
Student Aid Index values 

History and transactions on nonsampled stui<;nts 
(experimental bias): 

1 . Frequency ot corrections 

2. Submittal of arrent transactions 

3. Magnitude of SAI changes established through 
corrections 



Corrections Control Group Ih 
Students at Nonsampled Schools 



History and transactions on students at non- 
sampled schools: 

1 . Frequency of corrections 

2. Conseqv«nces of corrections in terms of SAI 
changes 



Institutional Master File 



1 . School characteristics 

2. Sampling information 



Student Validation Roster 



Final recoiKiled disbursements 



FIGURE 3-1 (continued) 

PELL GRANT QUALITY CONTROL DATA SOURCES, 
COLLECTION MODES AND TYPES OF DATA 
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CaMs with 

Diff«r«noes Cases With Diitercnces 

PossUily Cases ' PosaiMy Produdiy Overpayment 

i^ppUcation Dependency Leading to Accurate "55 SiOl szoi 5301 swl 5501 

Item Status Underpayment Within $5 $100 $200 $aOO iHQO $500 $600 $6004- 



Adjusted 
Gross Income 



Independent 


26 


k75 


17 


6 


6 


2 


3 


2 


29 


Dependent 


46 


1,038 


19 


8 






3 


2 


19 


Total 


72 


1,513 


36 


U 


11 


2 


6 




48 



Taxes Paid 



Independent 


28 


1*53 


IS 


23 


9 


6 




4 


3 


Dependent 


62 


977 


22 


19 


10 


5 


11 


3 


16 


Total 


90 


1,«30 


40 


43 


19 


11 


15 


7 


19 



Parental 
Social 
Security 
Benefits 

Dependent 19 21 3 2 1 1 1 - 7 



FIGURE A-1 

DOCUMENTED DIFFERENCES BETWEEN SAR VALUES AND 
DOCUMENTED VALUES: NON-EXEMPT CASES 
FLAGGED FOR VALIDATION 
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with 
Dilforenccs 

Application Dependency Lcatfingto Accurate "55 
Item Status Underpayment Within $5 $100 



Cases With Differences 
Possibli 




Adjusted 
Cross Income 





IndepefKtent 


i 


1 1*9 


I* 


2 






1 


8 






O 


235 


3 


3 


I 


1 




8 




Total 


16 




7^ 


5 


1 


1 


1 


16 


Tastes Paid 


Independent 


9 


1<»6 


6 














Dependent 


11 


225 


2 


6 


1 


1 




1 




Total 


20 


371 


8 


6 


I 


1 




1 


Parental 
Social 
Security 
Benefits 






















Dependent 


S 


t* 








I 


1 


2 



FIGURE A-2 

DOCUMENTED DIFFERENCES BETWEEN 5AR VALUES AND 
DOCUMENTED VALUES: NON-EXEMPT CASES 
NOT FLAGGED FOR VALIDATION 
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A. NON-MONETARY ITEMS 



Documwiteu Vattie htt 



Total 



than 
SAR 



Equal 
to 
SAR 



Greater 
than 
SAR 



with 
Documen- 





A{^iicaticMi Item 


Value 


Value 


Value 






Household Size 




100% 




8 




Number of Exemptions 


2% 


91% 


6% 


386 




Number in Postsecondary Education 




100% 




8 


B. 


INCOME ITEMS 


Pferocfit Where QgosnsRfsd 
Vkluc Less SAR Value bs 


Total 
Cases 




AppUcation Item 


Zero 
or 

Less 


$1 
to 
$200 


$201 
or 
Greater 


with 
Documen- 
tation 




Father /Student Income Portion 


78% 


11% 


11% 


126 




Mother/Spouse Income Portion 


76% 


10% 


14% 


91 




AFDC/ADC 


77% 




19% 


48 




Dependent Student's Income 


48% 


17% 


35% 


46 




Student Social Security 
Educational Benefits 


78% 


5% 


17% 


18 




VA Educational Benefits 


100% 






3 




All Other Income and Benefits 


23% 


47% 


30% 


SO 


C 


DEDUCTIONS 


Perotint Where Documented 
Value Less SAR Value Isx 


Total 
Cases 




Application Item 


Zero 
or 
Greater 


-$1 
to 
-$200 


-$201 
or 

Less 


with 
Documen- 
tation 




Medical/Dental Expenses 


71% 


13% 


15% 


52 



FIGURE A- 3 

PRESENCE OF DOCUMENTATION AND DIFFERENCES 
FROM SAR VALUES FOR NOT FLAGGED CASES: 
SELECTED APPLICATION ITEMS* 

♦The following items were documented less than 0.2 percent of the time and therefore were 
not included in this figure: home assets, business assets, investment assets, cash and 
savings; dependent student's assets, and unreimbursed tuition. 
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APPEMMX B 
SAIinJNG METHODOLOGY 



RATIONALE 

The primary objective of the sample design was to choose a probability sample of 
Pell Grant recipients enrolled at educational institutions participating in the 1982-83 
Pc!l Grant program. The statistically representative sample was used to document the 
rate at which institutions complied with the 1982-83 Pell Grant vaadation 
requirements. To do this, data was obtained from student financial aid files at 
educational institutions. In spring, 1983, this sample wUl be used to document, 
compute, and analyze the program-wide error rates. Data in the spring will be 
collected from students and their parents, as well as from institutions. 

An important goal in the sample design was to organize available information for 
drawing the sample in order to reduce the variability of the characteristics of possible 
samples. One design feature used to achieve this goal was sample selection with 
probability proportional to a measure of size (MOS) that was presumed to be 
correlated with the statistics being estimated. In the institution stage of sample 
selection, the measure of size was the number of Pell Grant recipients reported by the 
institution in the October 1981 progress report. The October 1981 progress report was 
used for two reasonsi (1) it satisfied the need for a sample of students receiving 
grants in the fall? and (2) current year (1982-83) progress reports were incomplete. ' 
Sample Size and Precision 

The choice of sample size was mainly determined by the budget and time 
constraints on the study. The maximum number of institutions that could be contacted 
in the six-week field period for the institutional data collection was about 300. 
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Based "on the data avaUable from Stage One of the Quality Control study and 
$<xne basic assumpti^ons on parameters that were not available at that time, the 
precfsion that could ^ expected with 300 institutions and an average of 12 students 
per institution was estimated. A student characteristic such as absolute dollar error 
(S) could be looked at as the product of two variables to be estimated trom the survey t 
the average absolute dollar value of the error given that an error was made (§) and the 
proportim of cases in error (P), thus 

S 3 P6 

The relvariance (s<^iare of the coefficient of variation) from a cluster sample was 
approximated by: 

V ^ ^ 1 ll > p (fi - 1) 



mft 



where 



2 1 

V = (i-P), relvariance, between students, of the amount of error; 

e 



2 

V = unit relvariance of the amount of error for those cases which are in 
e error; 

P = oroportion of cases in error; 

m s: number of cli^ters; 

p = intra-cluster correlation coefficient; and 

n = average cluster size. 

Estimates of the coefficient of variation (CV) for several values of P are 

presented in Figure B-1. The following parameter values are assumed: 

p = .5 (variance estimates from Stage One suggest that the intra-class 
correlation between students m a school is quite high); 

V = i (derived from Stage One variance estimates); 



mn = 3600. 
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FiGURfiB-i 

EXPECTED COEFFICIENT OF VARIATION FOR AVERAGE ABSOLUTE DOLLAR 
VALUE OF ERROR, ASSUMING DIFFERENT VALUES OF P 





Proportion of studenti < 


who had son 


Be error 




P X .« 


P « .6 


Pa.* 


P « .2 


Coefficient of variation 


.052 


.065 


.085 


.127 



Design Ob jectives 

In terms of survey implementation, the proposed sample design had the following 
objectives: 

• Limit the amount of field travel; 

• Control the number of stud^ts and separate institutions selected; and 

• Insure the representation of a variety of institution types. 

The procedures described below yielded a sample meeting these objectives. 

INSTITUTION SAMPLE: SCHOOLS USING REGULAR DISBURSEMENT 

SySTEM( RDS) \ 

Sampling Frame 

The sampling frame for the RDS institution sample was constructed from the 
Pell Gr^t Program Institution Universe file and the October 1981 Progress Report 
Error/Unreasonable file. The following steps were taken in the construction of the 
frame. 

Selection of a Single Report Per Institution 

A singie report per institution was extracted from the Progress Report (PR) file 
which contained progress reports for October and Ad Hoc or update reports. The Ad 
Hoc report was retained if one was available, otherwise, the initial October report was 
retained. Branches of institutions reporting through a central office had no PR record 
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because the central office accounted for their Pell Grant recipients. Details on the 
treatment of multi-branch campuses where the central office reports for all branches 
are given in a section to follow. 
Eiigible Institutions 

After a single PR record was extracted from each institution, these records were 
merged with the Universe (U) file, by institution ID, and only in-scope institutions 
were retained in the frstme. For purposes of this survey, institutions were considered 
in-scope ifs 

• They were in the conterminuous United States (excludes Puerto Rico, 
Alaska, Hawaii and the Virgin Islands). 

• Had RDS participation code—position 2<fO on Universe file—of either 1 
(participating, independent campus), 2 (central office for participating 
tM-anch campus system), or 3 (branch campus participating through a 
central^office). 

• Had eligibility code— position 239 on Universe file— of 1 (eligible for Pell 
Grant). 

• Had institutional static code— position ^^9^ on Universe file— of 1 (active 
institution which may eft may not be funded). 

Imputation of Recipients (Measure of Size) 

Eligible institutions which were on the U file but not on the PR file, other than 
those reporting through a central office, were flagged for imputation of the number of 
lecipients. Two hundred and forty-two institutions were in this group. Based on ^,676 
institutions that had data on the number of recipients, undergraduate enrollment, 
institution type, and institution control, a regression function to predict recipients as a 
function of the other three variables was estimated, in the estimation, dummy 
variables for institution type and control categories were generated and the most 
general model, which included two-way and three-way interactions, was used. The R^ 
obtained was .72 (proportion of the variance in the dependent variable explained by the 
regression); a very good fit for cross-section data. Sixty-three institutions had no 
repoited recipients and no undergraduate enrollment figure and thus imputation with 
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the regression function was not possible. These institutions were listed for inspection, 
and since they were fairly small schools (such as "Pedigree School of Dog Grooming"), 
they were assigned a value of S recipients to be used as a measure of size. 

Treatment of Campuses where the Central Office Reports for all Branches 
Institutions where the central office reported for all the branches (Participation 
Code = 2)--the individual branches had no PR record and data on October recipients- 
were handled as follows: 

• For campuses broken <town by branches in the "Education Directory - 
Colleges and Universities 19S2-83," the recipients reported by the central 
office was allocated <unong the main campus and branches according to 
their respective enrollments; that is: 

Recipients^ = (Enrollmentg/Total Enrollment) 
X Total Recipients 

• For other instituti(Mis, if the central office was selected, the school was 
called and a determination was made if records of Pell Grant recipients 
were kept at the central office or at the individual branches. If records 
were kept at the c^itrai office, students from the central office files were 
sampled, and individual hranches were not distinguished. 

• For institutions that kept records at the individual branches, we obtained 
the breakdown of number of participants by branch and selected one branch 
within the institution, probability proportionate to size (PPS), using the 
recipients at the branch as t\\e measure of size. 

Determination of Certainty Institutions 

To obtain a final sample of about 300 cooperating sdK>ols, allowing for 2 percent 
out-of-scope institutions and 95 percent institution cooperation, required an initial 
sample of 322 institutions. Institutions with reported recipients greater than the 
overall institution selection interval (total measure of size/322) were drawn into the 
sample with certainty. 

The certainty cutoff was set at 3,7(»0 or approximately 70 percent of the school 
selection interval. That is. 

Total Recipients x .70 = 1 7»6,131 x .70 = 37*^0 
School Sample 322 
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This certainty cutoff yielded 3* certainty institutiww, <^ of which were central offices. 
Seiec^on of Nonoertainity Institutions 
Design Overview 

The sample design for the noncertainty portion of the sample was basically a 
(k)ubie sampling procedire with PPS selecticMi of clusters and implicitly stratified PPS 
selection of all schools included in the sample dustc The sample design called for: 

• Ordering the file by geographic code and forming clusters of consecutive 
schools from the ordered frame of about 8 schools each. 

• Sampling clusters with probability proportional to total recipients in the 
cluster 

• Assigning an adjusted measure of size (AMOS) to schools within sampled 
clusters where 

AMOS.. = IMOS^/CMOS. 

i ^ school; 

j 35 cluster; 

IMOS = the institution measure of size; and 

CMOS = the cluster measure of size. 

• Ordering the schools in sample clusters, ignoring the cluster identifier, by 
the following type-control strata: 

School control 

Public 

Private 

Proprietary 

School type 

Offering I to 2-year programs 
Offering 2 to 3-year programs 
Offering <^year (or more) programs 

• Sampling schools systematically from the ordered file with probability 
proportional to the adjusted measure of size (AMOS)* 
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Zip Racode 

After obtaining a frame for the noncertainty portion of the sample in which all 
institutions had a value for the measure of size and recipients, it was merged with the 
zip-cocte-recode file. The zip-code-recode step attached a serpentine geographic code 
(GEOCODE) to each institution and removed invalid (those that did not match the 
master list) three-digit zips for verification and correction. 

Formation of Cl uster 

To limit the amount of field travel, instituticms were clustered in geographically 
contiguous three-digit zips. The file was sorted by GEOCODE and a cluster was 
defined as a set of S consecutive schools with a total minimum measure of size to 
insure an average of 12 recipients per institution. The minimum measure of size was 
achieved by all clusters. The clustering process resulted in 632 clusters of & schools 
each, with an average of 2,<*70 recipients pa- cluster. 

Sample of Clusters 

Out of the 632 ciiKters in the frame, 72 clusters were sampled, PPS, with total 
number of recipients in the cluster as the measiffe of size (CMOS). The 72 sampled 
clusters contained 576 schools. None of the clusters' sizes exceeded the sampling 
interval of 21,713 which would have required their selection with certainty at this 
stage. 

Sample of Institutions 

To ensure adequate representation of different institution types in the second 
stage of sampling, institutions were stratified by control (public, private and 
proprietary) and type (less than 2 years, 2 to 3 years, and years or more). The 
measure of size for a PPS selection was the ratio of the institution's MOS to the 
measure of size of the cluster from which the institution came (CMOS). That is, 

AMOS.. = IM0S.-/CMOS| 
as defined in an earlier section. The 576 institutions in the 72 sample clusters were 
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sorted by the type and control strata defined above and a systematic sample of 2S8 
institutions was selected with probability proportional to the adjusted measure of size 
(AMOS). A considerable number of institutions (171 or 39.3%) had a measure of size 
larger than the sampling interval and were conditional certainties at this state; these 
were removed from the frame before the remaining 117 noncertainty institutions were 
drawn. 



Distribution of the Sample Among Type-Control Strata 

As a result of the sample design, the final distribution of the sample among type- 
control strata was close to proportional to the distribution of the universe measure of 
size among the strata. Figure &-2 below compares twth distributions. Clearly the 
design goal was satisfactorily achieved. 



RGURE B-2 



COMPARISON OF DISTRIBUTION OF RECIPIENTS IN 
INSTITUTION UNIVERSE TO SAMPLED INSTITUTIONS 



Institution P Uc Private Proprietary Total 



Type 


Value 


Percent 


Value 


Percent 


Value 


Percent 


V^iUe 


Percent 


Less than 2 yrs. 
Universe MOS 
Sample Count 


10,343 
2 


.6 
.6 


3,126 


.18 


92,704 
22 


5.3 
6.8 


106,173 
24 


6.1 
7.5 


2-to-3 years 
Universe iMOS 
Sample Count 


451,443 
88 


25.9 
27.3 


34,663 
10 


1.9 
3.1 


46,416 
14 


2.7 
4.3 


532,522 
112 


30.5 
34.7 


4 years or more 
Universe MOS 
Sample Count 


759,500 
111 


43.5 
34.5 


342,776 
74 


19.6 
23.0 


5,160 
1 


.3 
.3 


1,107,436 
186 


57.8 


Total 
Universe iMOS 
Sample Count 


1,221,286 
201 


70.0 
62.4 


380,565 
84 


21.7 
26.1 


144,280 
37 


8.3 
ii.4 


1,746,131 
322 


100.0* 
100.0* 



* Rounded 



59 

B-8 

ERIC 



SELECTION OF STUDENTS FROM SAMPLE RDS WSTITimONS 

To obtain a sample M about 3,600 PeU Grant recipients (300 institutions with an 
average of 12 students per school) required an initial sample of about k,OW recipients, 
assuming a 90 percent combined institution/recipient response rate. The overall 
recipient sampling fraction required to achieve an equal probability sample of the 
desired size is given by; 

i = "I = »0»0 = 1 

Tmos 17<>6131 ^35:51 



where 



f = overall sampling fraction; 

TMOS = total number of recipients (total measure of size) computed from 
the October 1981 progress report file; 

"l = the required initial sample of recipients. 
Consequently, the weight associated with each sample recipient, the reciprocal of the 
overall sampling rate, is ^32.21 

The overall sampling fraction in conjunction with the first- and second-stage 
sampling fractions specified by the sample design yielded a specific within-institution 
"ampling fraction as follows: 

i - f^ X fj X f^j = 1 J 



432.21 



thus 

^WI " 1 X j_ X 1_ 

432.21 f f, 
c I 

where 

^WI ' recipient sampling fraction within sample institution; 

f^ = sampling fraction of the cluster from which the institution came; and 

f J = institution sampling.! raction within a sample cluster. 
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Sampling Frame 

The sampiing frame was the list of Pell Grant recipients, at each of the sample 
institutions, from which the recipient sample was drawn. At the time the school was 
contacted to set up an appointment for the data collector's visit, the school was 
requested to prepare a list of current and pending recipients to be used as sampling 
frame. 

Whether the list was available or the data collector had to compile it, special 
care was taken to insure that no Pell Grant recipient was knowingly excluded. Also, 
the list was checked for duplicates. 
Selection Procedure 

Recipients were selected from the sampling frame systematically with equal 
probability. For this purpose a sampling worksheet was produced for each institution. 
The sampling worksheet included all the information necessary for drawing the sample, 
such as sampimg interval and pregencrated selectidn numbers. 

After the data collectors obtained a list that included all Pell Grant recipients 
eligible for the study, they numbered sequentially the students on the list. Then, they 
selected those students with sequence numbers corresponding to the selection numbers 
given on the sampling worksheet. 

WEIGHTED ESTIMATES 
Stuctents 

The sample of Pell Grant recipients from RDS schools was an equal probability, 
self-weighting sample; before adjustments for nonresponse, all students had the same 
weight. 

Let X. be the value of a variable x for the i^^ student, and w^ the weight 
associated with that stu<ient. After adjustment for nonresponse, an estimate of the 
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universe total for the x variable is given by: 

^ = I ^'i'^i 
isl 

where 

.th 



- weight associated with the i student after nonresponse adjustment. 

Y can simUarly be defined for another variable, y. In particular, if y takes the 
value of 1 for every student, the ratio X/Y estimates either an average or a proportion 
depending on whether X is an indicator variable or a multi-valued variable. For. 
example, if X is the student's income, X/Y is the average income of Pell Grant 
recipients. 
Institutions 

The use of institution estimates is not recommended, especially for subsets of 
institutions, because the sample was designed to estimate student characteristics. 
However, institution estimates can be generated as follows: 

• Let Xij be the value of a characteristic of the i^h institution in the jth 
cluster at the first stage of selection. (Certainty institutions may be 
considered in a "certainty cluster" in which all institutions in the cluster 
were drawn into the sample with certainty.) 

• The weight (after nonresponse adjustment) associated with the ith 
institution in the jth cluster is wjj. 

• An estimate of an institution total, X, is given by 

X = ^ W..X 

ij 
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APPENIMXC 

SUMMARY OF COMMENTS FROM FINANCIAL AID ADMINISTRATORS 

AND DATA COLLECTORS 

During the field work, many financial aid administrators (FAAs) offer^id their 
complaints and suggestions to the data collectors who visited their institutions. The 
following comments stem from two soirees. The first are omcerns voiced by FAAs 
during t,.»e sampling and compUing of data in the field. The other, a summary of the 
comments provided by the data collectors during a debriefing held on December 1 1, 
19S2. The comments have been organized by the following five topicsi 

• Administrative Problems 

• CommiHiicatlon Problems with ED Regarding all Issues, Including 
Validation 

• Problems with the Pell Grant Application Process 

• Problems with Validati<^ Procedt«-e$ 

• Recommendations to Improve Validation Procedures 
Overview 

The majority of FAAs who commented from the sampled institutions 
acknowledge the positive effects of the expanded validation effort, however, they feel 
that the many problems and delays caused by costly, inconsistent, and unrefined 
validation procedures cannot be overlooked. A variety of factors, ranging from 
overburdened financial aid office staffs and inadequate commtnication with ED, to 
difficulty in obtaining documentation from government agencies, contribute to the 
dissatisfaction of many FAAs. These complaints and suggestions are targeted at 
program inefficiency and not at the merits of the validation effort. In fact, many 
FAAs consider the problems associated with increased validation inevitable and 
actively seek to resolve them. One FAA's remark that "the new validation is a burden, 
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however, it is acceptable if it does provide more money for the neediest students" 
sums up the attitude of many. It is important to note that FAAs who in the past 
validated 100 percent of their students had significantly fewer complaints and reacted 
positively to the expansion of validation. 

During the debriefing, the data collectors reinforced and expoimded upon the 
opinions expressed by FAAs concerning expam^ validation. Some of the data 
collectors reported that small sdK>ols could handle 100 percent validation but that 
large schools were overwhelmed.' Others, however, found that "it depends very much 
on the personality of the financial aid officer and their efficiency ... on how they 
view their job." Many institutions were already doing something very similar to 
validation for all their aid recipients to control institutional funds (especially at 
private institutions) or in response to the requirements of state programs. Many 
proprietary schools, aware that they are suspected of waste or fraud because of a few 
well-known cases, were especially scrupulous. Only one college was reported to 
disagree fundamentally with validation. As one data collector observed, "I don't think 
their argument is with validation itself. 1 . . . pinned them down on it . . . it's the 
process. They all, I think, see the need of validation." 

Generally, FAAs stress the need for more money to alleviate administrative 
burdens, for dear and explicit guidelines to eliminate inconsistent regulations and 
forms, and for timeliness of ED changes and announcements. As one FAA asserted: 
"Application and validation procedures result in declining student participation," 
hence, undermining the goals of the Pell Grant program. Because problems of 
validation ultimately harm student applicants, FAAs see the need for immediate 
improvements. 
Admlmstratlve Problems 

Despite the general acceptance of expanded validation, many FAAs found costi 
and administrative burdens excessive. There has been a reported increase in the 



amount of papenarork and the cost which the Institutions must bear in order to 
complete validation. Many administrators report that this is placing a substantial 
financial burden on the institutions. There is "concern over the increasing expense to 
institutions f or . . . validations, etc FAAs agree with the need to validate, but may 
have dUficulty in fimding positions to take care of the work if it continues to 
increase." 

There is strong interest among administrators in providing a cost allowance for 
validation and decreasing the overall cost to the institutions. Some FAAs fear that 
the money which ED provides for administration of the Pell Grant program is not being 
used for its intended purpose. "ED should specify that any administrative allowance 
paid to schools must be allocated to the direct benefit of the financial aid office." 
FAAs suggest "institutions be reimb»jrsed for students who are processed, but receive 
no award." 

In addition, some FAAs are wondering if the expense of validation has "proved to 
be justified by dollar changes on SARs." They are uncertain if much money has been 
saved by validating because of the expense of reprocessing and the cost to institutions. 

Additional cost allowances would not be helpful in all instances. Reporting on 
both their personal experience as financial aid administrators and on their conversa- 
tions in the field, the data collectors generally agreed that the financial aid office 
itself often does not benefit directly from the administrative cost allowance. This 
allowance "goes into institutional money and it may go to the library, or it may go to 
the business office, it docs not go back to the aid office." Even designating that the 
Federal money must be used only for administration of financial aid programs would 
not help since "there is no aid office I know of that can exist on Federal payment..." If 
the administrative cost allowance were increased, institutional funding would be 
reduced in proportion. Nevertheless, increasing the administrative cost allowance 
would reduce the overall institutional burden and obviate complaints by upper-level 
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administratofs that the government was imposing prodedures for which it would not 

pay- 
Communication Problems with ED Regarding ail Issues, Including Validation 

Aside from increased administrative burdens iH'ought on by expanded vaiidationi 
the FAOs who commented diring the study revealed much concern over inadequate 
commimication between ED and themselves* One FAA remarked that he could not 
^^accttf'ately inform stucfents and the community with any certainly about validation^ as 
a result of confusing proceditf'es outlined by ED. Several of the FAAs from sampled 
schools were critical of ^mnges made by ED which were not effectively transmitted 
to them* One FAA echoed the sentiment of many in his statement that "institutions 
must receive, in a timely manner, all information including payment schedules, 
validation handbooks, etc* • to avoid unnecessary delays which prove to be detri- 
mental to students* A member of the field staff concurred: think a big 
help . . . would be for the Department of Education to get their handbooks and their 
schedules out in a timely manner . . . that would improve validation more than any 
other factor." 

More specifically, changes in regulations by ED were considered untimely and 
confusing by many FAAs* Despite the long range benefits promised by expanded 
validation, FAAs were confronted with immediate problems leading to waiting and 
frustration by staff and students. For example, in one financial aid office "^payment 
schedule revisions resulted in a loss of activity by students in the office due to 
frustration because services were not provided when originally promised*" Another 
FAA observed that "mid-year regulation changes caused confusion and unnecessary 
burdens on the FAAs, and negatively affected service to students," while one FAA 
added that "the changes are frustrating and are contributing to burnout and rapid FAA 
turnover*" 
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This observation is supported by one dau collector who reported that the morale 
of even the best-informed FAAs has been devastated by frequent, rapid changes in 
regulations. Another of the field staff reported "a trainer, who had withcfr-awn as 
being a trainer because ... he couldnt accurately represent from minute to minute 
what the student aid process was because it was changing so frequently, and he said, 
. . I cant go out there and tell people this is what you should do when the next week 
is going to be different or the previous week had a change I dont know about.' 

Many felt that issuance of retroactive changes by ED must be avoided. One FAA 
suggested giving FAAs an 18 month lead time for regulation changes which would 
allow for staff and student adjietment and would lessen confusion considerably. 

The magnitude of problems arising from regulation and policy changes varied 
among the FAAs at sampled schools. Those who already validate 100 percent seemed 
to adjust easily to ED'S changes and did not report any adverse effects on students. 
The situation was different and more serious for other FAAs and students. One FAA 
claimed that "lateness of regulation changes conflicted with progress reports, and 
changes came so late that the situation made the colleges look bad. Holding up 
payments forced some students to stay out of school." Most FAAs agreed that 
providing "simple, English versions of ED changes" will expedite the IR efforts. 

The Validation Handbook is another example of communication cited for 
improvement by FAAs at many of the sampled institutions. Above all, FAAs stress 
earlier delivery of the Validation Hf.ndbook so they will have ample time to adjust to 
changes and to inform applicants.. As for the content of the handbook, comments 
focused on its "confusing" and "difficult" nature. 

Many felt the Validation Han(fi>ook inadequately outlines the extent of the 
responsibility of the FAA for validation. As one FAA explained; "The Validation 
Handbook needs to make clear exactly which items must be validated beyond AGl and 
taxes." Apparently, FAAs are confused about which categories they must document, 
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and which laii under •'optional data.** On« administrator beUeved that since ♦^he 
school is responsible for all discrepancies between College Scholarship Service's 
Financial Aid Form (FAF) and the SAR, the handbook should state just that facts all 
facts and figures must be checked." Clarification of the FAAs role in validation wUl 
eliminate much uncertainty, and consequently, wUl refine the Pell Grant deUvery 
process. 

A specific illustration of the need for improved communication C-in the use of 
the tax return. A number of FAAs suggested that ED provide some type of training on 
the use of the tax retian in Pell Grant validation. There is a perceived need for . . 
continuing in-service training for vaUdation. . . emphasizing help in reading and 
understanding the lQttQ,» Also, it has been requested that any workshops which ED 
sponsors be given . . after new forms are issued, not before." 

The comments made during the debriefing accentuated the problems of FAAs 
further. Based on their interviews, most of the data collectors agreed that training 
that is offered is not sufficiently publicized or regarded as particularly effective. One 
of the data collectors had concluded from his field interviews that "forms are getting 
more sophisticated, FAAs are being asked to make^'all kinds of judgements, become tax 
experts, and yet there is little training offered by the Federal government . . 
Training and communications from ED were considered less reliable than private 
information* . . members of NASFAA . . . felt they had the most constant interpre- 
tation of the regulations and ... the most security within their own operation about 
how to deal with validation ... The schools that depended on the Department of 
Education did not feel secure with what they were doing." 

Also, FAAs believed that there was a lack of support for their institutions by ED 
as a result of poor communication. Several concrete examples of this emerged at the 
debriefing of the data collectors. FAA credibility was damaged by conflicting policy 
or regulatory interpretations by ED staff: a student who was told one thing by his 



FAA could get a different answer by calling ED. Payment options were another sore 
point. Schools that were not in a financial position to take Option 2 tell their students 
that they must complete validation before receiving th^=- Grants, but the students then 
call ED and are told that the regulations permit the school to release the money in 
advance. Apparently it is not made clear to these students that the school has a right 
to excercise Option 1. Schools also complain that if they "found some real reason for 
concern that the applicatiwi was deliberately filed inaccurately they would 
generally . . . give the student the opportimity to make the corrections needed and 
say, 'If you do not, then we will be required to turn this over to the Department of 
Education . . But cases referred to ED are never prosecuted, so "it's sort of an 
empty threat." 

Problems With the Pell Grant Application Process 

Comments made by FA As concerning the Application fw Federal Student Aid 
and the Student Aid Report (SAR) touched on many aspects of the application process. 
There are several areas in which there is strong agreement among the sampled 
institutions as to changes which can be implemented to improve this process. These 
areas include: the need for more explicit directions for the applicants, the need for 
more comprehensive instructions for FAAs, and improvement of the edit system. 

The comment most frequently made by FAAs was that ' the directions for 
applicants should be much ore explicit. "Most students don't read and understand the 
application. . . Most errors found are from misinterpretation of instructions—very 
seldom from deliberate attempts to defraud." For example, more comprehensive 
instructions for FAAs are requested so that ". . . the division of assets for separated or 
divorced parents or applicants filing joint returns can be determined. . .," as well as 
other items for validation such as "ail other income and benefits" and the student's 
dependency status, 
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During the debriefihg, some of the data collectors highlighted similar problems 
students encountered with the Pell Grant application instructions. They considered it 
especially important to emphasize that students should report taxes paid rather than 
taxes witiihekl, a common error. Some students confuse Social Security benefits with 
social service benefits such as AFDC, so that too is an area which needs emphasis. 
The comments section of the SAR is another area where instructions could use 
improvement. "Students do not read it." Therefore, the FAAs would like the comment 
about taking the lO'^O to the financial aid office "in a block in bold type" separated 
from other items. One of the data collectors suggested the the "last two paragraphs 

are usually extraneous — there are only a couple of key paragraphs " For fi ling 

out the application, some FAAs suggested "a line-by-line correspondence rather than 
having to think about it. 3ust say 'take this line from the attached return and put it 
here.' " 

It was noted that ". . . more clarity is needed in defining independent student 
status. Distinction between dependent and independent students, based on the 
previous year, is too limiting. Some students use the parents' address simply as a mail 
drop." This point of view was echoed in concern about perceived ". . . inequities to 
many independent students living in the parents' home, but paying rent. More 
questions about dependency status should be in the application." It was suggested that 
consideration be given to the idea of requiring the tax forms for the previous two 
years in order to determine the student's dependency status. Dependency status is a 
point of confusion since FAAs find it difficult to categorize students living with 
relatives other than their parents. In this situation it is also difficult to determine the 
student's financial strength. 

The problems with identifying dependency status were also mentioned at the 
debriefing. In fact, one of the data collectors thought that "the financial aid 
administrators I spoke to were probably most concerned about the abuse of 
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independent status. The current definition hurts these students who live at home but 
get no support- Yet it seems unfair that there are many students whose parents make 
very large incomes who, after one year in college, will declare themselves independent 
and get three years of aid in independent status." ^ 

FAAs ask that ED "make reporting requirements for income stricter." There is a 
problem with "... income in IRAs, deductions, and nonreporting of capital gains. 
People with business Values' of zero, such as consultants, may actually have large 
incomes which become small taxable incomes due to deductions. This makes it 
difficult for their eligibility (for Federal student aid) to be ascertained." 

FAAs want much clearer instructions for the reporting of other specific income 
items. "Clearer instructions are needed for Social Security benefits. There should be 
differentiation between all kinds of benefits: SSI, disability, etc. The definition of 
disability income for item 27C of the application should be given in order to avoid 
confusion with Social Security disability benefits." 

The confusion over completing the application has caused some students to 
hesitate filling out the form. "Many students fear filling out the application. They are 
afraid of the consequences of a mistake. They also fear that the information which 
they provide will not be held confidential by the application processor." One FA A 
said, "The federal financial aid system and its assortment of forms . . . discourage 
some students from applying for financial aid." 

A change which could facilitate the application process is refining .ne edit 
system. The majority of FAAs who comn>ented considered the SAR reprocessing 
delays to be one of the largest problems they face. One FAA described the turn- 
around time for SARs as "excessive, particularly with increased validation," while 
another FAA claimed "delays in reprocessing are not worth the small changes in money 
awards." Most FAA's comments focused on the adverse effects of reprocessing delays 
on students, and acknowledge the unfortunate result that "often low-income families 
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fail out of the Peil Grant process because of difficulty in processing forms." One 
financial aid office "receives the same edit problem over and over" thus increasing the 
delays in processing their students' awards. At another institution, the FAA found the 
central processor "often provides inconsistent, misleading, and incorrect advice to 
schools, that does not conform to regulations," thus perpetuating delays in processing. 
According to many FAAs who commented, some students complained that the central 
processor even lost their SARs. As expected, the "unreasonable" burdens placed on 
schools and students by these delays and errors prompted suggestions from FAAs to 
"reinstate the toll-free number" for questions, and to "switch to the College 
Scholarship Service (CSS) or similar processors to handle operations." 

Some FAAs chose to comment on how the processing problems affected their 
role. One FAA believed that "FAAs are now perceived by parents and students as 
barriers to education, not as helpers, as a result of needless delays and conflicting 
policies." One FAA went so far as to say that "the delays in reprocessing are 
deliberate attempts to discredit FAAs because of rhe resulting confusion." Despite 
differing opinions about the origins and effects of SAR reprocessing delays, a generally 
negative attitude prevails. 
Problems with Validation Procedures 

According to data collectors at the debriefing, the sheer amoun*- of time 
required for validation proved to be an obstacle for some students. If the Pell Grant is 
included in a financial aid package with money from other programs, such as 5EOG, 
having to return the SAR to the processor can delay the whole package by six to 
twelve weeks. By then, the school has reallocated the Campus-Based or institutional 
funds and the student has to turn to the GSL program for the additional money needed. 

Many FAAs suggested that ED continue to concentrate validation efforts on 
broad scale items such as AGI and U.S. taxes paid. Because ED's perspective is much 
wider, they realize the importance of every possible validation category, however, 
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many FAAs place a high priority only on the categories which they believe most 
profoundly affect students' awards, and thus set priorities accordingly. For example, 
one FAA commented that "medical and dental expenses rarely have an effect (on 
student's award); therefore, we never validate it." From the viewpoint of many FAAs, 
validation is a means to correct and prevent gross discrepancies, and not to be used as 
a control or policing mechanism for relatively minor ones. 

Some field representatives added that concentrating on high-priority items 
reduces the burden on the institutions. In one case last year, an award was held up for 
a student from Iran because the item for Veterans education benefits had been left 
blank, although it is difficult to conceive how such a student could have any VA 
benefits. Concentrating on "abuses that are upfront, like independent status," is a 
better investment of time and money than worrying about items that are rarely in 
error or would not affect the award. 

However, it was revealed at the debriefing that some FAAs simplified validation 
procedures to the point of noncompliance. The data collectors observed that "some 
schools when they do find a discrepancy . . . will recalculate it but they won't write 
anything down to document it." At many institutions, only corrections that affected 
the aid index or the amount of the award were noted on the SAR. 

It was suggested by a number of FAAs that an attempt be made to " . . . end the 
confusion over interest income exclusion and the SAR tolerances. Since SAR 
tolerance is $300 (for dependent students), the interest exclusion of $^00 caused many 
SAR errors. Therefore, many cases had to be reprocessed. This misunderstanding is 
costly and could be avoided by making SAR tolerance and interest exclusion 
compatible .... The expense of reprocessing a tolerance of $300 is not cost 
effective . . . . " 

Government agencies, namely the Social Security Administration (SSA), the 
Internal Revenue Service (IRS), and the Veteran's Administation (VA), are targets of 
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blame for delays in the validation process. Because "government agencies such as 
Social Security do not provide documents with alacrity to schools," and an "average 
turnover time for the IRS is four to five weeks," many FA As at the sampled 
institutions were legitimately frustrated. The repercussions of late documentation 
from such agencies ultimately reaches students, many of whom cannot afford delays in 
obtaining Pell Grant funds. 

Field representatives reported that Federal agencies caused other problems with 
validation besides mere delay in responding to information requests. SSA would not 
give information directly to the institutions, but gave the wrong information to 
students; nor would it use forms that one state developed for its public institutions to 
validate Social Sercurity benefits. Some local offices of the Immigration and 
Naturalization Service (INS) refused to provide copies of alien registration cards while 
others cooperated. Local welfare agencies seem to be particularly notorious for long 
delays in documenting welfare benefits received by students. 

Many FAAs also noted the untimeliness of Social Security regulation changes. 
As with other cases of misinformation, FAAs found they had an increased workload 
and more delays in disbursing Pell Grants as a result of retroactive changes. Cases 
wherein a student was determined to be eligible for Social Security benefits after 
already having been processed were cited by some FAAs. This type of problem 
contributes to much program inefficiency and is burdensome for everyone i-.volveu'. 
One FAA asserted that it is "the regional Social Security offices which do not inform . 
local offices of regulation changes," and consequently, tie up information needlessly. 

Regulation changes regarding Veteran's benefits contributed further to delays in 
disbursing Pell Grants. The greatest point of contention was that regulation changes 
created administrative problems and kept some students out of school. Retroactive 
changes which made some veterans eligible (and others ineligible) in mid-year were the 
chief source of complaints. "Some students stayed out of school for a semester— many 
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were dissatisfied.- Problems were created with documentation saying a veteran was 
originally eligible for a Pell Grant, but was declared ineligible due to a change in 
regulations. The mid-year regulation change caused confusion and resentment among 
veterans receiving educational benefits." 

Many veterans find it difficult to understand why Veterans benefits are shown as 

J 

income "while Job Corp and CETA income are not reported. . ." This difficiilty 
creates an area of conflict with veterans who perceive inconsistency of policy in the 
different criteria for reporting income. Several FAAs suggested Veterans Benefits 
again be included in the calculation of the SAI. The exclusion of Veterans Benefits 
from the calculation is being seen as "a reduction in the (Pell) Grant." 

Along with delays caused by regulation changes, FAAs had problems obtaining 
and using documentation. These problems stem from a i. umber of causes including! 
documentation which is incomplete, lack of documentation, and documentation which 
some FAAs find insubstantial. A good deal of the comments addressed the use of the 
10^*0 and lO^OA. Many FAAs suggest ED require the 10<>0, lO'fOA, or alternative as 
part of the application for Federal student aid. 

FAAs also discussed some of the problems which they are having in obtaining 
documentation, and the amount of emphasis placed on the documentation obtained 
from tax return filers versus nonfiiers. It was stressed that there is a need to have 
. . . nonfiiers sign a notarized statement regarding the amount of their nontaxable 
income in addition to the statement of nonfiling." Some FAAs suggested to our data 
joUectors that a statement of nonfiling be included as a section of the application. 
Institutions then would not have to devise their own forms and go through a special 
procedure for students who had not filed. One FAA asked: "Why not verify all 
income such as ADC? The tax filers are looked at much more closely than nonfiiers." 

Institutions ir» the Southwest are having problems in obtaining documentation 
because the parents and/or spwuses of many students are employed in Mexico. This 
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leads to the problem ot converting the income amount to U.S. dollars when documen- 
tation is obtained. There are also problems, on a more general basis, with obtaining 
documentation from students with parents residing abroad. 

There is also the problem of documenting a student's past financial aid. It has 
been suggested that the matter of the financial aid transcript be addressed. FA As find 
themselves in a peculiar situation since they have no way of knowing if a student 
attended another school and received financial aid. "The student in default at one 
school can still get a Pell Grant somewhere else. Even if a student defaulted on a loan 
at another school, the present school cannot hold up the Pell payment." 

Finally, FAAs have commented that many families do not keep documents. 
Students from low-income families often do not keep documentation and therefore 
cannot complete validation. This is of particular concern to FAAs since they want to 
assist in getting money to the neediest students. 

Such difficulties in obtaining documentation contribute to abuses of the Pell 
Grant program. Several particular abuses emerged during the debriefing of the fi>ld 
staff. For one, "it was very easy to cheat if students just signed the ^taiement of 
nonfiling because the FAA had very little recourse to find out whether or not they 
actually had income." One financial aid director contacted the IRS after it transpired 
during interviews that a nonfiler had considerable income, but the IRS "just couldn't do 
anything . . . just didn't give them any names." At a school in the midwest, "claiming 
that parents are deceased" is "the latest scam"; the financial aid director wants to ask 
for df.ath certificates next year. In general, "the more you institutionalize validation 
the more institutionalized the cheating may become . . . The more we do this on a 
regular basis, the more we ask it of everybody, the more we're getting systematic 
cheating." Another data collector reported that some FAAs feel that "before, the 
presumption was that the students were honest and . . . some f>eople . . . are ripping us 
off, but now the presumption is that students are dishonest and now we're trying to 
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prove that they are honest . . . they feel the cheating has actually gone up because 
people . . . feel they are expected to cheat." 
Recommendations to Improve Validation Proceckires 

One session of the debriefing was devoted to recommendations for improvement 
of validation procedures. There was general agreement on the usefulness of simply 
requiring every student to submit a copy of his tax return (and his parents', if 
dependent) with the SAR. Some FA As thought that the lO^^O should be submitted to 
the processor (central or MDE) with the application and checked there; the institutions 
should not have to collate the application and tax data. Some schools suggested that 
they should be "certified and be able to do (validation) themselves . . . they would like 
to work like SEOG and get audited afterwards if they mess up." (Schools in California, 
for instance, have to compile comparable data for their state programs.) Validation 
would be smoother if they could just do it once. 

A second discussion focused around incentives for institutions. Under current 
regulations, an institution which discovers overawards saves the Federal government 
money, but does not help the institution or its honest or accurate applicants. Allowing 
the institution to keep some of the savings for administrative costs would provide an 
incentive for doing 100 percent validation and doing it well. 

A third suggestion was to combine extensive validation for a small sample of 
students with validation of only a small number of items—no mqre than four— for 
everyone else. Many of the items that are supposed to be validated, such as assets, 
either cannot be done at all or are so difficult as to be impractical to do universally. 

Other suggestions cited during this debriefing session h«ve been incorporated 
elsewhere in the appendix. 
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